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Alan Turing's 
rorgotten 
ceas 


in 
Computer Science 


Well known for the machine, 
test and thesis that bear his name, 
the British genius also anticipated 

neural-network computers 
and “hypercomputation” 


by B. Jack Copeland and Diane Proudfoot 


Alan Turing’s Forgotten Ideas in Computer Science 


lan Mathison Turing conceived of the modern 
computer in 1935. Today all digital comput- 
ers are, in essence, “Turing machines.” The 
British mathematician also pioneered the field of 
artificial intelligence, or AI, proposing the famous 
and widely debated Turing test as a way of determin- 
ing whether a suitably programmed computer can 
think. During World War II, Turing was instrumental 
in breaking the German Enigma code in part of a 
top-secret British operation that historians say short- 
ened the war in Europe by two years. When he died 
at the age of 41, Turing was doing the earliest work 
on what would now be called artificial life, simulat- 
ing the chemistry of biological growth. 

Throughout his remarkable career, Turing had no 
great interest in publicizing his ideas. Consequently, 
important aspects of his work have been neglected or 
forgotten over the years. In particular, few people— 
even those knowledgeable about computer science— 
are familiar with Turing’s fascinating anticipation of 
connectionism, or neuronlike computing. Also ne- 
glected are his groundbreaking theoretical concepts 
in the exciting area of “hypercomputation.” Accord- 
ing to some experts, hypercomputers might one day 
solve problems heretofore deemed intractable. 


The Turing Connection 


D igital computers are superb number crunchers. 
D Ask them to predict a rocket’s trajectory or calcu- 
late the financial figures for a large multinational cor- 
poration, and they can churn out the answers in sec- 
onds. But seemingly simple actions that people routine- 
ly perform, such as recognizing a face or reading 
handwriting, have been devilishy tricky to program. 
Perhaps the networks of neurons that make up the 
brain have a natural facility for such tasks that standard 
computers lack. Scientists have thus been investigating 
computers modeled more closely on the human brain. 

Connectionism is the emerging science of computing 
with networks of artificial neurons. Currently research- 
ers usually simulate the neurons and their interconnec- 
tions within an ordinary digital computer (just as engi- 
neers create virtual models of aircraft wings and 
skyscrapers). A training algorithm that runs on the 
computer adjusts the connections between the neurons, 
honing the network into a special-purpose machine 
dedicated to some particular function, such as forecast- 
ing international currency markets. 

Modern connectionists look back to Frank Rosen- 
blatt, who published the first of many papers on the 
topic in 1957, as the founder of their approach. Few re- 
alize that Turing had already investigated connectionist 
networks as early as 1948, in a little-known paper enti- 
tled “Intelligent Machinery.” 

Written while Turing was working for the National 
Physical Laboratory in London, the manuscript did not 
meet with his employer’s approval. Sir Charles Darwin, 
the rather headmasterly director of the laboratory and 
grandson of the great English naturalist, dismissed it as 
a “schoolboy essay.” In reality, this farsighted paper 
was the first manifesto of the field of artificial intelli- 
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gence. In the work—which remained un- 
published until 1968, 14 years after Tur- 
ing’s death—the British mathematician 
not only set out the fundamentals of con- 
nectionism but also brilliantly introduced 
many of the concepts that were later to 
become central to AI, in some cases after 
reinvention by others. 

In the paper, Turing invented a kind of 
neural network that he called a “B-type 


be accomplished by groups of NAND 
neurons. Furthermore, he showed that 
even the connection modifiers themselves 
can be built out of NAND neurons. 
Thus, Turing specified a network made 
up of nothing more than NAND neu- 
rons and their connecting fibers—about 
the simplest possible model of the cortex. 
In 1958 Rosenblatt defined the theo- 
retical basis of connectionism in one suc- 
cinct statement: “Stored 


information takes the 


Few realize that Turing 
had already investigated 
connectionist networks 
as early as 1948. 


unorganized machine,” which consists of 
artificial neurons and devices that modify 
the connections between them. B-type 
machines may contain any number of 
neurons connected in any pattern but are 
always subject to the restriction that each 
neuron-to-neuron connection must pass 
through a modifier device. 

All connection modifiers have two 
training fibers. Applying a pulse to one 
of them sets the modifier to “pass 
mode,” in which an input—either 0 or 
1—passes through unchanged and be- 
comes the output. A pulse on the other 
fiber places the modifier in “interrupt 
mode,” in which the output is always 
1, no matter what the input is. In this 
state the modifier destroys all informa- 
tion attempting to pass along the con- 
nection to which it is attached. 

Once set, a modifier will maintain its 
function (either “pass” or “interrupt”) 
unless it receives a pulse on the other 
training fiber. The presence of these inge- 
nious connection modifiers enables the 
training of a B-type unorganized ma- 
chine by means of what Turing called 
“appropriate interference, mimicking 
education.” Actually, Turing theorized 
that “the cortex of an infant is an unor- 
ganized machine, which can be orga- 
nized by suitable interfering training.” 

Each of Turing’s model neurons has 
two input fibers, and the output of a 
neuron is a simple logical function of its 
two inputs. Every neuron in the net- 
work executes the same logical opera- 
tion of “not and” (or NAND): the out- 
put is 1 if either of the inputs is 0. If 
both inputs are 1, then the output is 0. 

Turing selected NAND because every 
other logical (or Boolean) operation can 
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form of new connections, 
or transmission channels 
in the nervous system (or 
the creation of conditions 
which are functionally 
equivalent to new connec- 
tions).” Because the de- 
struction of existing con- 
nections can be func- 
tionally equivalent to the creation of new 
ones, researchers can build a network 
for accomplishing a specific task by tak- 
ing one with an excess of connections 
and selectively destroying some of them. 
Both actions—destruction and creation— 
are employed in the training of Turing’s 
B-types. 

At the outset, B-types contain random 
interneural connections whose modifiers 
have been set by chance to either pass or 
interrupt. During training, unwanted 
connections are destroyed by switching 
their attached modifiers to interrupt 
mode. Conversely, changing a modifier 
from interrupt to pass in effect creates a 
connection. This selective culling and en- 
livening of connections hones the initially 
random network into one organized for 
a given job. 

Turing wished to investigate other 
kinds of unorganized machines, and he 
longed to simulate a neural network and 
its training regimen using an ordinary 
digital computer. He would, he said, “al- 
low the whole system to run for an ap- 
preciable period, and then break in as a 
kind of ‘inspector of schools’ and see 
what progress had been made.” But his 
own work on neural networks was car- 
ried out shortly before the first general- 
purpose electronic computers became 
available. (It was not until 1954, the year 
of Turing’s death, that Belmont G. Farley 
and Wesley A. Clark succeeded at the 
Massachusetts Institute of Technology in 
running the first computer simulation of 
a small neural network.) 

Paper and pencil were enough, though, 
for Turing to show that a sufficiently 
large B-type neural network can be 
configured (via its connection modifiers) 


in such a way that it becomes a general- 
purpose computer. This discovery illumi- 
nates one of the most fundamental prob- 
lems concerning human cognition. 

From a top-down perspective, cogni- 
tion includes complex sequential process- 
es, often involving language or other 
forms of symbolic representation, as in 
mathematical calculation. Yet from a 
bottom-up view, cognition is nothing but 
the simple firings of neurons. Cognitive 
scientists face the problem of how to rec- 
oncile these very different perspectives. 

Turing’s discovery offers a possible so- 
lution: the cortex, by virtue of being a 
neural network acting as a general-pur- 
pose computer, is able to carry out the se- 
quential, symbol-rich processing dis- 
cerned in the view from the top. In 1948 
this hypothesis was well ahead of its 
time, and today it remains among the 
best guesses concerning one of cognitive 
science’s hardest problems. 


Computing the Uncomputable 


Jn 1935 Turing thought up the ab- 
| stract device that has since become 
known as the “universal Turing ma- 
chine.” It consists of a limitless memory 


Turing’s Anticipation 
of Connectionism 


| n a paper that went unpublished 
| until 14 years after his death (top), 
Alan Turing described a network of 
artificial neurons connected in a ran- 
dom manner. In this “B-type unorga- 
nized machine” (bottom left), each 
connection passes through a modifi- 
er that is set either to allow data to 
pass unchanged (green fiber) or to de- 
stroy the transmitted information (red 
fiber). Switching the modifiers from 
one mode to the other enables the 
network to be trained. Note that each 
neuron has two inputs (bottom left, in- 
set) and executes the simple logical 
operation of “not and,” or NAND: if 
both inputs are 1, then the output is 
0; otherwise the output is 1. 

In Turing’s network the neurons in- 
terconnect freely. In contrast, modern 
networks (bottom center) restrict the 
flow of information from layer to layer 
of neurons. Connectionists aim to 
simulate the neural networks of the 
brain (bottom right). 
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that stores both program and data and 
a scanner that moves back and forth 
through the memory, symbol by sym- 
bol, reading the information and writ- 
ing additional symbols. Each of the ma- 
chine’s basic actions is very simple— 
such as “identify the symbol on which 
the scanner is positioned,” “write ‘1’” 
and “move one position to the left.” 
Complexity is achieved by chaining to- 
gether large numbers of these basic ac- 
tions. Despite its simplicity, a universal 
Turing machine can execute any task 
that can be done by the most powerful 
of today’s computers. In fact, all mod- 
ern digital computers are in essence 
universal Turing machines [see “Turing 
Machines,” by John E. Hopcroft; Sc1- 
ENTIFIC AMERICAN, May 1984]. 
Turing’s aim in 1935 was to devise a 
machine—one as simple as possible— 
capable of any calculation that a human 
mathematician working in accordance 
with some algorithmic method could 
perform, given unlimited time, energy, 
paper and pencils, and perfect concen- 
tration. Calling a machine “universal” 
merely signifies that it is capable of all 
such calculations. As Turing himself 
wrote, “Electronic computers are in- 


tended to carry out any definite rule-of- 
thumb process which could have been 
done by a human operator working in a 
disciplined but unintelligent manner.” 

Such powerful computing devices 
notwithstanding, an intriguing question 
arises: Can machines be devised that are 
capable of accomplishing even more? 
The answer is that these “hyperma- 
chines” can be described on paper, but 
no one as yet knows whether it will be 
possible to build one. The field of hyper- 
computation is currently attracting a 
growing number of scientists. Some 
speculate that the human brain itself— 
the most complex information proces- 
sor known—is actually a naturally oc- 
curring example of a hypercomputer. 

Before the recent surge of interest in 
hypercomputation, any information- 
processing job that was known to be 
too difficult for universal Turing ma- 
chines was written off as “uncom- 
putable.” In this sense, a hypermachine 
computes the uncomputable. 

Examples of such tasks can be found 
in even the most straightforward areas 
of mathematics. For instance, given 
arithmetical statements picked at ran- 
dom, a universal Turing machine may 
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not always be able to tell which are the- 
orems (such as “7 + 5 = 12”) and which 
are nontheorems (such as “every num- 
ber is the sum of two even numbers”). 
Another type of uncomputable problem 
comes from geometry. A set of tiles— 
variously sized squares with different 
colored edges—“tiles the plane” if the 
Euclidean plane can be covered by 
copies of the tiles with no gaps or over- 
laps and with adjacent edges always the 
same color. Logicians William Hanf and 
Dale Myers of the University of Hawaii 
have discovered a tile set that tiles the 
plane only in patterns too complicated 
for a universal Turing machine to calcu- 
late. In the field of computer science, a 
universal Turing machine cannot always 
predict whether a given program will 
terminate or continue running forever. 
This is sometimes expressed by saying 
that no general-purpose programming 
language (Pascal, BASIC, Prolog, C and 
so on) can have a foolproof crash de- 
bugger: a tool that detects all bugs that 
could lead to crashes, including errors 
that result in infinite processing loops. 
Turing himself was the first to investi- 
gate the idea of machines that can per- 
form mathematical tasks too difficult 
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TOM MOORE (illustrations); KING'S COLLEGE MODERN ARCHIVES, CAMBRIDGE UNIVERSITY LIBRARY (top); PETER ARNOLD, INC. (bottom right) 


the Uncomputable 


ple, an oracle might work. 


EXCERPT FROM TURING’S THESIS 


problea. Wore definitely 


A Turing proved that his universal machine—and by ex- ; 
tension, even today’s most powerful computers—could - 
never solve certain problems. For instance, a universal Turing 
machine cannot always determine whether a given software 
program will terminate or continue running forever. In some 
cases, the best the universal machine can do is execute the 
program and wait—maybe eternally—for it to finish. But in his 
doctoral thesis (below), Turing did imagine that a machine 
equipped with a special “oracle” could perform this and other 
“uncomputable” tasks. Here is one example of how, in princi- 


Consider a hypothetical machine for solving the formidable 


Let us suprose that we are supplied with some unspecifiod 
mans of solving suster theoretic probless; a kind of araale as it 
mere. We will not go any further into tha stuss of this oracle 
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Using an Oracle to Compute 


m 


100001...001111 


BINARY REPRESENTATION 


OF PROGRAM 


COMPUTER PROGRAM 


“terminating program” problem (above). A computer pro- 


gram can be represented as a finite string of 1s and Os. This 
sequence of digits can also be thought of as the binary rep- 


resentation of an integer, just as 1011011 is the equivalent 


of 91. The oracle’s job can then be restated as, “Given an in- 


are to 


teger that represents a program (for any computer that can 
be simulated by a universal Turing machine), output a ‘1’ if 
the program will terminate or a ‘0’ otherwise.” 

The oracle consists of a perfect measuring device and a 
store, or memory, that contains a precise value—call it t for 
Turing—of some physical quantity. (The memory might, for 
example, resemble a capacitor storing an exact amount of 


for universal Turing machines. In his 
1938 doctoral thesis at Princeton Uni- 
versity, he described “a new kind of ma- 
chine,” the “O-machine.” 

An O-machine is the result of aug- 
menting a universal Turing machine 
with a black box, or “oracle,” that is a 
mechanism for carrying out uncom- 
putable tasks. In other respects, O-ma- 
chines are similar to ordinary com- 
puters. A digitally encoded program is 


chine—for example, “identify the sym- 
bol in the scanner”—might take place.) 
But notional mechanisms that fulfill the 
specifications of an O-machine’s black 
box are not difficult to imagine [see box 
above]. In principle, even a suitable B- 
type network can compute the uncom- 
putable, provided the activity of the neu- 
rons is desynchronized. (When a central 
clock keeps the neurons in step with one 
another, the functioning of the network 

can be exactly simulat- 


ed by a universal Turing 


fed in, and the machine produces digital 
output from the input using a step-by- 
step procedure of repeated applications 
of the machine’s basic operations, one 
of which is to pass data to the oracle 
and register its response. 

Turing gave no indication of how an 
oracle might work. (Neither did he ex- 
plain in his earlier research how the ba- 
sic actions of a universal Turing ma- 
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Even among experts, Turing’s 

pioneering theoretical 
concept of a hypermachine 
has largely been forgotten. 


machine.) 

In the exotic mathe- 
matical theory of hyper- 
computation, tasks such 
as that of distinguishing 
theorems from nonthe- 
orems in arithmetic are 
no longer uncomput- 
able. Even a debugger 
that can tell whether any program writ- 
ten in C, for example, will enter an 
infinite loop is theoretically possible. 

If hypercomputers can be built—and 
that is a big if—the potential for crack- 
ing logical and mathematical problems 
hitherto deemed intractable will be 
enormous. Indeed, computer science 
may be approaching one of its most sig- 
nificant advances since researchers 


wired together the first electronic em- 
bodiment of a universal Turing machine 
decades ago. On the other hand, work 
on hypercomputers may simply fizzle 
out for want of some way of realizing 
an oracle. 

The search for suitable physical, 
chemical or biological phenomena is 
getting under way. Perhaps the answer 
will be complex molecules or other 
structures that link together in patterns 
as complicated as those discovered by 
Hanf and Myers. Or, as suggested by 
Jon Doyle of M.LT., there may be natu- 
rally occurring equilibrating systems 
with discrete spectra that can be seen as 
carrying out, in principle, an uncom- 
putable task, producing appropriate 
output (1 or 0, for example) after being 
bombarded with input. 

Outside the confines of mathematical 
logic, Turing’s O-machines have largely 
been forgotten, and instead a myth has 
taken hold. According to this apoc- 
ryphal account, Turing demonstrated in 
the mid-1930s that hypermachines are 
impossible. He and Alonzo Church, the 
logician who was Turing’s doctoral ad- 
viser at Princeton, are mistakenly credit- 
ed with having enunciated a principle to 
the effect that a universal Turing ma- 
chine can exactly simulate the behavior 
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chines “fall outside Turing’s concep- 
tion” and are “computers of a type nev- 
er envisioned by Turing,” as if the 
British genius had not conceived of such 
devices more than half a century ago. 
Sadly, it appears that what has already 
occurred with respect to Turing’s ideas 
on connectionism is starting to happen 


ORACLE'S MEMORY WITH T= 0.00000001101... 


PROGRAM 
8,7354 = r ——_ WILL all over again. 
NOT 
DECIMAL EQUIVALENT : 
Q TERMINATE The Final Years 
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Jn the early 1950s, during the last 
| years of his life, Turing pioneered the 
field of artificial life. He was trying to 
simulate a chemical mechanism by 
which the genes of a fertilized egg cell 
may determine the anatomical structure 
of the resulting animal or plant. He de- 
scribed this research as “not altogether 
unconnected” to his study of neural net- 
works, because “brain structure has to 
be ... achieved by the genetical embry- 
ological mechanism, and this theory 
that I am now working on may make 
clearer what restrictions this really im- 
plies.” During this period, Turing 
achieved the distinction of being the first 
to engage in the computer-assisted ex- 


ORACLE 


electricity.) The value of t is an irrational number; its written representation would 
be an infinite string of binary digits, such as 0.00000001101... 

The crucial property of t is that its individual digits happen to represent accu- 
rately which programs terminate and which do not. So, for instance, if the integer 
representing a program were 8,735,439, then the oracle could by measurement 
obtain the 8,735,439th digit of t (counting from left to right after the decimal 
point). If that digit were 0, the oracle would conclude that the program will process 
forever. 

Obviously, without t the oracle would be useless, and finding some physical vari- 
able in nature that takes this exact value might very well be impossible. So the search 
is on for some practicable way of implementing an oracle. If such a means were found, 
the impact on the field of computer science could be enormous. —B.J.C. and D.P. 
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ploration of nonlinear dynamical sys- 
tems. His theory used nonlinear differ- 


of any other information-processing ma- 
chine. This proposition, widely but in- 
correctly known as the Church-Turing 
thesis, implies that no machine can carry 
out an information-processing task that 
lies beyond the scope of a universal Tur- 
ing machine. In truth, Church and Tur- 
ing claimed only that a universal Turing 
machine can match the behavior of any 
human mathematician working with 
paper and pencil in accordance with 
an algorithmic method—a considerably 


weaker claim that certainly does not rule 
out the possibility of hypermachines. 
Even among those who are pursuing 
the goal of building hypercomputers, 
Turing’s pioneering theoretical contribu- 
tions have been overlooked. Experts 
routinely talk of carrying out informa- 
tion processing “beyond the Turing lim- 
it” and describe themselves as attempt- 
ing to “break the Turing barrier.” A re- 
cent review in New Scientist of this 
emerging field states that the new ma- 


ential equations to express the chem- 
istry of growth. 

But in the middle of this groundbreak- 
ing investigation, Turing died from 
cyanide poisoning, possibly by his own 
hand. On June 8, 1954, shortly before 
what would have been his 42nd birth- 
day, he was found dead in his bedroom. 
He had left a large pile of handwritten 
notes and some computer programs. 
Decades later this fascinating material is 
still not fully understood. 
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